UVeQFed: Universal Vector Quantization for Federated Learning

Traditional deep learning models are trained at a centralized server using data samples collected from users. Such often include private information, which the users may not be willing to share. Federated (FL) is an emerging approach train such without requiring share their data. FL consists of iterative procedure, where in each iteration copy model locally. The then collects individual updates and aggregates them into global model. A major challenge that arises this method need user repeatedly transmit its learned over throughput limited uplink channel. In work, we tackle tools quantization theory. particular, identify unique characteristics associated with conveying rate-constrained channels, propose suitable scheme for settings, referred as universal vector (UVeQFed). We show combining methods yields decentralized training system compression induces only minimum distortion. theoretically analyze distortion, showing it vanishes number grows. also characterize how conventional federated averaging combined UVeQFed converge minimizes loss function. Our numerical results demonstrate gains previously proposed terms both distortion induced accuracy resulting aggregated